Extracting the collective wisdom in probabilistic judgments

Abstract How should we combine disagreeing expert judgments on the likelihood of an event? A common solution is simple averaging, which allows independent individual errors to cancel out. However, can be correlated due overlap in their information, resulting a miscalibration average. Optimal weights for weighted averaging are typically unknown and require past data estimate reliably. This paper proposes algorithm aggregate probabilistic under shared information. Experts asked report prediction meta-prediction. The latter average other individuals’ predictions. In Bayesian setup, I show that if consistent estimator, percentage predictions meta-predictions exceed same. An “overshoot surprise” occurs when two measures differ. Surprising Overshoot uses information revealed overshoot surprise correct prediction. Experimental evidence suggests performs well moderate large samples aggregation problems where individuals disagree